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1,  Introduction 

The  purpose  of  thl*  paper  Is  to  alscuss  the  application  of 
'^dynamic  programming  techniques  to  a  ej.ass  o:  problem.:-  uhich  i-o. 
'^want  of  a  better  term  ^^call®''^ccrr.cir.atorial .  Tne  essential 
difficulty  of  these  problems,  from  tne  stanapcinv-  of  the 
analyst »  lies  In  their  apparent  lack  of  complexl\:y.  Usually, 

It  Is  either  a  question  of  performing  a  finite  set  of  arith- 
nstle  operations  or  determlnins  the  largest  of  a  ilr.lte  set  wi 

mnnbers. 

If  there  are  one  hundred  elements  in  the  finite  so«,  v;e 
'  oan  classify  the  problem  as  trivial.  If,  houever,  the  finite 
•'Lsat  possesses  a  million  members,  or  a  h'andrcd  million,  -v.  — 

E"  worthwhile  to  ask  whether  or  not  there  are  more  efficient  tech- 
^nlques  than  Just  an  element-by-element  onaminatlon . 

Problems  of  this  nature  arise  in  the  follov;inG  ’.:ays: 

1,  Solving  linear  systems  of  equations  oi  tne  *orm 

'  ‘i'  "  “  " . . 


2.  Maximization  of  a  linear  form  L(x)  - 

constraints  of  the  form 


C  C 


i  ‘>1' 


>  •  •  •  > 


1  =  1,  2 
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'Phe  purpose  of  this  paper  Is  to  discuss  the  application  of 
dynamic  programming  techniques  to  a  class  of  problems  which  for 
want  of  a  better  term  we  call  combinatorial.  The  essential 
difficulty  of  these  problems,  from  the  standpoint  of  the 
analyst,  lies  In  their  apparent  lack  of  complexity.  Usually, 

It  Is  either  a  question  of  performing  a  finite  set  of  arith¬ 
metic  operations  or  determining  the  largest  of  a  finite  set  of 
numbers . 


If  there  are  one  hundred  elements  In  the  finite  set,  we 
can  classify  the  problem  as  trivial.  I^,  however,  the  finite 
set  possesses  a  million  members,  or  a  hundred  million.  It  Is 
wortliwhlle  to  ask  whether  or  not  there  are  more  efficient  tech¬ 
niques  than  Just  an  element— by— element  examination. 

Problems  of  tills  nature  arise  in  the  following  ways: 

1.  Solving  linear  systems  of  equations  of  the fbrm 

■  h'  1  2 . "• 


2.  Maximization  of  a  linear  form  L(x) 
to  constraints  of  tiie  form 


N 

3  c.x.  sul Ject 
1-1  ^  ^ 


^  '^1'  1  -  1.  2. 


3.  Maximization  of  functions  over  finite  sets,  sucii  as 


permutations,  paths  along  a  grid,  and  so  on. 
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At  the  present  time,  there  Is  no  systeiratlc  theory  of 
problems  of  this  genre,  nor  Is  It  likely  that  there  ever  will 
be,  considering  the  mamy  varieties  and  sources.  There  are, 
however,  some  categories  of  problems  recognized  as  tractable. 
Some  are  soluble  explicitly  In  traditional  analytic  terms,  some 
by  means  of  algorithms  that  can  he  carried  out  by  hand,  amd 
some  require  the  most  powerful  computers  available. 

In  what  follows,  we  shall  discuss  various  ways  In  which 
the  functional  equation  technique  of  dynamic  programming  can  be 
applied.  We  shall  use  only  those  portions  of  the  general 
theory  required  for  our  present  purposes,  referring  the  reader 
Interested  in  further  aspectn  to  our  book. 

Although  we  shall  not  present  any  specific  numerical 
results  here,  we  shall  furnish  references  to  extensive  compu¬ 
tational  studies  carried  out  by  S.  Dreyfus  and  the  author. 
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COMBINATORIAL  PROCESSES  AND  DYNAMIC  PROQRAMMINO 

Richard  Bellman 


1.  Introduction 

The  purpose  of  thle  paper  Is  to  discuss  the  application  of 
dynamic  programming  techniques  to  a  class  of  problems  which  for 
•rant  of  a  better  term  we  call  combinatorial.  The  essential 
difficulty  of  these  problems,  from  the  standpoint  of  the 
analyst,  lies  in  their  apparent  lack  of  complexity.  Usually, 

It  Is  either  a  question  of  performing  a  finite  set  of  arith¬ 
metic  operations  or  determining  the  largest  of  a  flnihv?  set  of 
numbers. 


If  there  are  one  hundred  elements  In  the  finite  set,  we 
can  classify  the  problem  as  trivial.  If,  however,  the  finite 
set  possesses  a  million  members,  or  a  hundred  million.  It  Is 
•rorthirhlle  to  ask  whether  or  not  there  are  more  efficient  tech¬ 
niques  than  Just  an  element-by— element  examination. 

Problems  of  this  nature  arise  In  the  following  ways: 

1.  Solving  linear  systems  of  equations  of  the  form 


N 


2.  Maxlialzatlon  of  a  linear  form  L(x) 
to  constraints  of  the  form 


N 

y  c.x,  subject 
1-1  ^  ^ 


1-1.2, 


M. 
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3.  Maximization  of  functions  over  finite  sets,  such  as 
permutations,  paths  along  a  grid,  and  so  on. 

At  the  present  time,  there  Is  no  systematic  theory  of 
problems  of  this  genre,  nor  la  It  likely  that  there  ever  will 
be,  considering  the  many  varieties  and  sources.  TYiere  are, 
however,  some  categories  of  problems  recognized  as  tractable. 
Some  are  soluble  explicitly  In  traditional  analytic  terms,  some 
by  means  of  ali^orlthms  that  can  be  carried  out  by  hand,  and 
some  requiring  the  most  powerful  computers  available. 

In  what  follows,  we  shall  discuss  various  ways  In  which 
the  functional  equation  technique  of  dynamic  programming  can  be 
applied.  We  shall  use  only  those  portions  of  the  general 
theory  required  for  our  present  purposes,  referring  the  reader 
Interested  In  further  aspects  to  our  book,  [l]  . 

Although  we  shall  not  present  any  specific  numerical 
results  here,  we  shall  furnish  references  to  extensive  compu¬ 
tational  studies  carried  out  by  S.  Dreyfus  and  the  author. 


2.  An  Allocation  Problem 

Let  us  begin  with  the  following  simple  allocation  problem. 
Suppose  that  we  have  a  quantity  x  of  a  resource  which  we  are 
going  to  subdivide  Into  N  parts,  x^ ,  x^,  ...,  x^,  corres¬ 
ponding  to  N  different  activities.  To  make  a  mathematical 
problem  of  this,  let  us  suppose  that  we  are  given  functions 

which  measure  the  return  from  the  1— th  activity  due  to 
an  allocation  x^. 

The  question  of  most  efficient  allocation  of  resources 


l«Ad8  to  the  analytic  problem  of  maximizing  the  function 
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( 1 )  ( X j  f  » •  •  • , )  •  S^Cx^^)  ^  gg ^ ^  ^  ^  gj^ ( Xjj ) 

•ubjeot  to  the  constraints 

(2)  (a)  ^2 

(b)  >  0. 

Although  this  may  seem  like  a  most  prosaic  problem,  and 
hardly  worth  any  attention  at  this  late  date  in  the  history  of 
oaleulus,  as  we  shall  see  it  has  its  hidden  pitfalls. 

The  r\in-of— the-4nlll  approach  to  this  problem  converts  it 
by  way  of  a  Lagrange  multiplier  into  that  of  maximizing  the  new 
function 

N  N 

(3)  0ji|( Xj  , X2 » •  •  • » x^j )  ■  ~  ^1^1^^* 

where  X  Is  a  parameter  that  will  be  detemlned  from  (2a). 

The  variational  equations  are 

(4)  ^.O.g'(Xj)  -  X,  1  .  1,  2 . N. 

Solving  these  N  equations  for  the  x^  in  terms  of  X, 

•  x^(x),  the  parameter  X  is  determined  by  means  of  the 

relation 


N 


(5) 


X. 
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Although  this  approach  la  Infallible  in  textbook  problema, 
a  number  of  difficulties  arise  In  applications.  Let  us 
enumerate  them. 

In  the  first  place,  the  functions  g^(x)  may  not  have  a 
derivative.  Although,  as  we  shall  see  below,  we  do  possess  *.ui 
efficient  technique  for  solving  a  maximization  problem  of  this 
type,  we  shall  not  Insist  upon  this  point.  It  is,  however, 
reasonable  to  expect  that  the  Individual  functions  need  not 
possess  derivatives  at  various  points. 

Let  us  Ignore  these  bizarre  possibilities  and  assume  that 
It  Is  sufficient  to  examine  the  solutions  of  (4)  and  (5).  If 
eaoh  of  the  functions  g^(x^)  is  monotone,  which  Is  to  say 
that  g^(x)  Is  either  convex  or  concave,  then  the  Inverse 
function  x^(x)  Is  uniquely  defined  and  It  becomes  wlatlvely 
easy  to  study  the  solutions  of  (5). 

Since  It  Is  quite  common  for  utility  functions  g|^(x)  to 
have  points  of  inflection.  If  we  wish  to  resolve  general  prob¬ 
lems  of  this  nature  we  must  consider  situations  In  which  the 
equations  in  (4)  have  a  multiplicity  of  solutions.  Assuming, 
for  the  sake  of  moderate  complication,  that  each  equation  of 
the  form  ^  possesses  two  30iuticns  for  any  particular 

value  of  X,  we  see  that  the  solution  of  any  equation  such  as 
(5)  leads  to  a  consideration  of  2  cases. 

This  number  2  arises  by  counting  all  possible  cases. 

The  problem  thus  appears  to  have  unpleasant  combinatorial 


overtones . 


P-1284 

2-24-58 


We  can  amplify  these  overtones  In  a  number  of  ways.  In 
the  first  place,  we  can  Insist  that  the  endpoints  of  the 
Xj— Intervals  be  tested.  The  value  -  0  has  a  very  Impor¬ 
tant  interpretation.  It  means  that  the  1-th  activity  Is  not 
engaged  In  at  all.  The  problem  of  taking  Into  account  all 
possibilities  of  end-point  extrema  greatly  complicates  the 
enumeration  of  cases. 

Secondly,  we  can  Impose  additional  constraints  of  the  type 


(6) 


■  0. 


The  meaning  of  a  constraint  of  this  type  Is  that  the  use  of  one 
activity  effectually  prevents  the  use  of  another. 

So  far,  we  have  been  complaining  about  the  limitations  of 
an  approach  based  on  calculus.  Let  us  further  curtail  this 
technique  by  allowing  the  x^  to  range  only  over  the  elements 
of  a  discrete  set.  Thus,  we  may  Impose  the  restriction 

(7)  1,  2,  <••,  !■!,  2,  ...,N. 

At  this  point  the  analyst  Is  tempted  to  feel  that  the 
cards  have  been  too  thoroughly  stacked  against  him.  Let  the 
computing  machines  take  over;  let  them  solve  the  prol lem  hy  the 
trivial  method  of  examining  all  possibilities. 

The  people  In  charge  of  the  computers,  h.owever,  may  become 
a  bit  aggrieved  at  this  attitude.  They  will  concede  that  they 
possess  fantastic  machines  operating  at  phenomenal  speeds  that 
can  resolve  In  a  matter  of  hours  problems  that  would  have  con- 
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Burned  llfetlmea  even  twenty  years  ago.  But  theae  problema  must 
be  carefully  choaen.  Even  rudimentary  problema  of  other  types 
cannot  be  solved  by  enumeration  of  cases. 

Plnce  this  statement  may  come  as  a  shock  to  anyone  who  has 
not  taken  the  trouble  to  compute  the  total  number  of  possible 
cases  arising  from  simple  combinatorial  problems,  let  us 
Illustrate  this  point  by  means  of  a  question  Involving  permu¬ 
tations  . 

Take  the  problem  of  placing  N  objects  In  N  pigeonholes, 
assuming  that  we  are  given  a  function  which  measures  the  value 
of  each  assignment  of  objects.  To  resolve  the  problem  by 
examining  all  cases,  we  must  evaluate  N!  different  cases, 
corresponding  to  all  different  permutations. 

Accustomed  as  we  are  to  the  familiar  function  N!,  we 
seldom  realize  Its  rapidity  of  growth.  For  N  ■  10,  we  ^)ave 
5,628,000  possibilities,  a  formidable  but  not  Incredible  numl)er. 
For  N  -  20,  It  will  amuse  the  reader  to  calculate  how  long  It 
would  take  a  computing  machine  which  could  evaluate  one  permu¬ 
tation  a  microsecond  to  examine  all  cases.* 

It  follows  that  the  mathematician  cannot  abdicate.  He  Is 
obligated  to  develop  algorithms  which  can  handle  these  strange, 
new  problems.  Our  aim  will  be  to  present  some  simple  algorithms 
which  are  particularly  suited  to  digital  computers.  This  Is  not 
to  be  considered  our  ultimate  objective,  but  merely  a  preliminary 
step  on  the  way  to  understanding. 

- 1 - 

A  simple  lower  bound  Is  half  a  million  yearsj 
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3»  Punctional  Bquatlons 

Let  us  now  present  an  approach  to  these  problems  quite 
different  from  that  of  the  calculus.  Litroduce  the  function 
defined  for  x  >  0  and  N  •  1,  2,  by  the 

relation 


(1) 


fj,(x) 


Max 


Where  Is  the  region  In 

the  relations 


,x^)— epace  defined  Ly 


(2)  (a)  -  X, 

(b)  x^  >  0. 


The  only  assumption  we  need  make  concerning  the  gj^(x)  Is 
that  they  are  continuous  for  x^  >  0.  In  the  cases  we  shall 
treat  below  where  the  assume  only  a  finite  set  of  values, 

•wen  this  restriction  will  not  be  necessary. 

Let  US  write* 

(5)  gp(Xp)  +  •••  +  g,.(x^)i, 

"  0<x^  ( x^ )  ^  ^  ^  ^  N  7J 


Where  ^  l^^^  defined  by  the  Inequalities 


(t)  (a)  +  •  •  •  ♦ 


^N-l 


X  — 


This  is  a  particular  application  of  the  "principle  of 

optlMillty,  see  [lit  P*  B3. 


p-i2b^ 


(b)  >  0, 


Hence ,  for  N  ■  2  ,  .  . . , 


(5) 


fj^(x)  -  Max 

<^x 


*  •••  *  Kn-i<*n-i> 


Thus,  referring  to  the  original  definition  of  the  sequence 


(6) 


f^(x)  - 


Max 

0<x^<x 


SnW’ 


N  -  2,  5, 


Por  N  -  1,  we  have 
(7)  f^(x)  -  g^(x) . 

4.  Discussion 

The  preceding  formalism  redudee  the  original  multidimen¬ 
sional  maximization  problem  to  a  sequence  of  one— dimensional 
problems.  The  practical  significance  of  this  fact  Is  that  we 
now  do  possess  a  feasible  technique  for  solving  these  pr'otlems 
by  direct  search  methods  using  digital  computers. 

In  this  way,  we  can  treat  a  numter  of  problems  arising  In 
mathematical  economics,  engineering,  and  operations  research. 
Th.e  computational  solutions  of  these  questions,  Joint  work  with 

5.  Dreyfus,  will  appear  In  book  form  In  the  near  futur»'. 
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If  we  attribute  some  structure  to  the  functions,  such  as 
linearity,  quadratic  character,  convexity  or  concavity,  the 
recurrence  relations  In  (3.5)  can  be  used  to  determine  the 
analytic  character  of  the  sequence  |fj^(x)  and  the  maximizing 
as  funcclons  of  x;  cf.  ^i]  ,  [2],  Karush,  . 
Alternatively,  a  structural  feature  such  as  concavity  can 
be  used  to  accelerate  greatly  the  machine  search  for  a  maximum, 
cf.  Kiefer,  [4i]  ,  Johnson,  [^  ,  Johnson  and  Gross,  [3^,  Kiefer, 
Applications  of  the  technique  are  contained  In  jti]  ,  . 

5.  An  Iwbedding  Process 

It  Is  Important  to  point  out  what  we  have  accomplished  by 
Mans  of  the  functional  equation  technique.  We  have  taken  a 
particular  problem  with  a  specific  value  of  x  and  N  and 
made  it  a  member  of  a  family  of  prollems,  continuous  In  x  and 

discrete  in  N. 

In  other  words,  we  have  Imbedded  a  particular  process  with¬ 
in  a  family  of  processes.  Oddly,  It  Is  easier  to  treat  the 
particular  process  by  consideration  of  the  whole  family  of 
processes,  than  It  Is  to  treat  the  process  by  Itself. 

Further  discussion  of  this  point  will  be  found  In  [ij  ,  [23^  . 

6.  Constraints 

Let  us  now  examine  the  effect  of  constraints  upon  the 
Mthod  outlines  In  ^4.  Suppose  that  we  Impose  the  additional 

constraints 


0  <  a^  <  Xi  <  b^. 


N 


(1) 


1-1,2, 


•  •  • 


P-12b^ 

?_24_58 

-10- 


In  addition  to  those  of  (2.2). 

It  Is  easy  to  see  that  the  relations  of  (4.5)  are  replaced 
by 


(2) 


fj,(x) 


Max 


<?  i- 


where  Is  the  region  determined  by  the  new  conditions 

(3)  (a)  <  bj^, 

(b)  ^  ^  *  *'^'*^1^  • 

In  the  definition  of  fjj(x),  x  Is  restricted  by  the  lower 
bound  a^ 

TTie  Interesting  thing  to  note  Is  that  whereas  In  the  usual 
approach  to  maximization  problems  additional  constraints  cause 
difficulties,  here  the  more  constraints,  the  simpler  the  compu¬ 
tational  task.  Additional  constraints  restrict  the  region  over 
which  each  variable  can  roam,  and  thus  simplify  the  search  for 
a  maximum.  We  shall  mention  this  again  lelow. 

7.  Constraints— Discreteness 

As  an  example  of  a  nastier  type  of  constraint,  consider 
the  problem  of  maximizing 

N 

(1)  P(x^  .x^,  .  .  .  ,Xjj)  - 

l‘"l 


subject  to  the  constraints 
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(2)  (a)  «  0  or  1, 

(b)  <  X. 

As  before,  we  obtain  the  recurrence  relation 

(5)  f„(x)  -  Max 

-  Max  gjj(l)  +  ■*“ 

The  computation  can  now  be  carried  out  by  hand  In  a  very 
simple  fashion.  Observe  that  this  Is  the  slnplest  type  of 
uaxlmlfstlon  problem  that  a  machine  can  perform. 

8.  Mutually  exclusive  Activities 

Let  us  now  complicate  matters  still  further.  In  addition 
to  the  restrictions  In  the  preceding  section,  let  us  Impose  the 

constraint 

(l)  ^1^1-fi  ■  1-1,2,  N  -  1. 


fN_l(x-x^ 


To  treat  this  prollem,  Introduce  the  sequence  of  functions 
of  two  variables,  fji^(x,y),  defined  by  the  relations 


ro(x.y)  •  Max  rgj(xj)  +  ^^(Xj)  + 

'Si  " 


p-.n(x^) 


Where 

by 


Is  now  the  region  In 


,x  )-space  defined 

N 
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(5)  (a)  -f  4  •  •  •  •►  Xjj  <  X, 

(b )  -  0,  1 , 

(c)  1-1,2,  ...,  N  -  1, 

(d)  Xj^y  -  0. 

The  quemtlty  y  Is  allowed  to  take  only  two  values,  0  or  1. 
Then  we  nave  the  recurrence  relation 

r 

(  ^  )  ^1^  ( ^  I  y )  “  Kax  j  ( x^, )  f  ^  ( X  Xj^ ,  Xj^ )  , 

V  L. 

where  Xj^  Is  sul  Ject  to  the  conditions 
(O  (a)  Xj^  -  0,  1, 

it)  <  X  , 

(c)  x^y  -  0. 

In  order  to  resolve  the  original  problem  we  must  compute 
thie  two  sequences  f|^(x,0),  rj^(x,l). 

It  Is  easily  seen  that 

('  )  fj^(x,0)  -  Max  gj^(l)  +  . 

^'^(x,!)  -  g,g(0)  ^  f'j|_^(x,0). 

The  two  sequences  |fj^(x,0)j  ,  irj^(x,l)  ,  can  thus  le 
determined  quite  simply. 
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9.  More  Constraints 

Returning  to  the  olmpler  problem  discussed  In  ^2,  let  us 
consider  the  problem  of  maximizing 

(^)  ^  ^  •••  -f  )  * 

subject  to  the  constraints 

(2)  (a)  -►  +  ••••►  Xj^  <  X, 

(b)  a^x^  -f  a^x^  >•••■»•  a^^Xj^  <  y,  >  0, 

(c)  >  0. 


Observe  that  we  have  replaced  equality  signs  ‘^y  Inequali¬ 
ties,  since  this  avoids  some  unimportant  consistency  require— 

■ants . 

Introducing  the  sequence  of  functions,  fwi(^‘.y)»  defined 

ri 

by 


(5) 


■  Max  P(x^  ^x^,  .  .  .  ,x^) 


for  H  ■  1,  2,  ...,  x,y  >  0,  It  Is  easy  to  see  that,  as  In 
the  preceding  sections,  we  obtain  the  recurrence  relation 


(*) 


■  >  a,  with 


(5) 


f j(x,y)  -  Max  g^(x^ )j  , 


V 
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for  0  <  [x,y/a^l  . 


10.  General  Formulation 

There  Is  no  difficulty  In  formulating  the  problem  cf 
maximizing 


(1) 


P^(X)  . 


N 


5e( 


X 


(1)  .(2) 


1-1 


,x(x)) 


subject  to  the  constraints 

(2)  (a)  1  yj>  J  -  1,  2, 

(b)  >  0, 


M 


In  the  same  fashion.  Setting 

(3)  ’^2' "  • > 


we  Bee  that 


Max 


g(  ^ 


4“’) 


-►  f 


N-1 


(1) 


Prior  to  any  discussion  of  the  computational  feaslMllty 
of  an  algorithm  of  this  type  for  general  values  of  M,  let  us 
turn  to  a  p>artlcular  problem  of  this  type  In  which  large  values 
of  M  enter  In  a  most  natural  way. 
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11.  *Rie  Hltchcock-Koopmana  Transportation  Problem 

L«t  UB  now  discuss  one  of  the  most  Interesting  models  In 
mathematical  economics,  the  Hitchcock— Koopmans  model  of  the 
flow  of  connodltles . 

Suppose  that  at  N  different  locations,  which  we  shall 
call  sources ,  there  are  quantities  of  an  Item  which  must  he 
transported  to  M  other  locations  which  we  shall  call  sinks . 

Let  denote  the  quantity  of  the  item  at  the  1— th 

source,  denote  the  demand  for  this  Item  at  the  J— th  sink, 

and  denote  the  cost  of  transporting  a  unit  quantity  from 

the  1— th  source  to  the  J— th  source.  Furthermore,  assume  that 
the  total  supply  at  the  sinks  Is  equal  to  the  total  demand  from 
the  sources. 

•nie  problem  Is  to  determine  a  shipping  policy  which  mini¬ 
mizes  the  cost  of  supplying  the  demand.  To  reduce  this  problem 
to  analytic  form,  let 

(l)  x^j  ■  the  quantity  sent  from  the  1— th  source  to 

the  J— th  source. 


Then  we  are  required  to  minimize  the  linear  function 

N  M 


(2) 


L(*)  -22  *11*11 

1-1  j-i 


ewer  all  x.  satisfying  the  linear  constraints 
^  J 

N 


<•)  j^*ij  *  *1’ 


N 

\ 
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M 


This  problem  Is  one  that  can  be  treated  very  successfully 
by  the  "simplex  technique"  of  0.  Dantzlg,  ,  or  by  the  newer 
methods  of  Pulkeraon  and  Ford,  l  also  Prager,  [4^.  Both 
of  these  methods  depend  strongly  upon  the  linearity  of  the 
various  equations. 

It  can  easily  be  shown  that  the  linearity  of  all  the 
functions  Involved  prevents  the  existence  of  any  Internal 
maximum.  The  region  defined  by  the  relations  >f  (5)  1#  the 
Interior  of  a  raultl-dlmenslonal  polyhedron.  To  decemlnc  the 
maximum  of  L(x),  It  Is  sufficient  to  examine  the  values  of 
L(x)  at  the  vertices  of  this  region. 

It  follows  that  we  have  a  problem  of  combinatorial  type. 
The  methods  described  above  furnish  efficient  search  techniques 
These  methods  fall  In  general  If  we  Introduce  nonlinear 
cost  functions.  We  shall  employ  functional  equation  techniques 

12.  The  Nonlinear  Transportation  Problem 

Let  US  examine  the  prol lem  of  minimizing 

(1)  g(x)  - 

over  all  satisfying  the  constraints  of  (10.3),  where  the 

gj^j(x)  are  not  necessarily  linear. 


To  treat  the  question  \y  means  of  functional  equations,  we 
can  proceed  In  one  of  two  ways.  To  begin  with,  assume  that  we 
satisfy  the  total  demand  In  the  following  fashion:  first,  the 
demand  of  the  M— th  sink,  then,  having  satisfied  this,  th.e 
demand  of  the  (M-l)— ot  sink,  and  so  on. 


0 

0 


‘IM 


y- 


^2M 


For  fixed  demands,  ,  let 


( 2 )  ’ ^2 ' *  *  * '  ^  * 


Then  the  same  reasoning 
equation 


the  minimum  cost  to  satisfy  t^ e 

demands  of  M  I'emalnlng  sinks, 

starting  with  quantities 

X.  ,  x_,  ...,  X.,  at  the  N  sinks. 

Id  N 

as  we  have  used  al ove  yields  the 


(5) 


f  |l|  ( X  ^  ,  x^  , 


^  - 


Min 

^IM 


u- 


N 


^M-1^^1  ^IM 


*2  ^2M 


^NM 


IM 


where  the  x 


\ttry  over  the  region  determined  ly 
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(b)  1“1,2,  ...,N. 

Tb.e  function  f  ^  (  x^  ,  x^  ,  .  .  .  , )  Is  given  by 

N 

(b)  ^  ^11^^11^* 

We  have  thus  transformed  the  original  pr  lem  into  that  of 
computing  the  sequence  |f j^(  , .  .  .  ,  Xj^  )j  . 

It  is  clear  that  we  could  obtain  an  altei native  formu¬ 
lation  by  using  first  all  the  resources  of  the  Fi— th  source  to 
satisfy  some  of  the  demands  at  the  M  sinks,  and  so  on. 

i;?.  Feasibility 

Let  us  now  see  whether  or  not  the  recurrence  relations  pre¬ 
sented  in  (12.3)  actually  lead  to  a  fehsitle  computational 
scheme.  At  eachi  stage  of  the  computation  we  have  to  tabulate  a 
function  of  N  variables,  and  perfonn  a  minimization  over  an 
N— dlraenslona 1  region. 

Although  both  of  these  are  formldal  le  procedures  if  M 

and  N  are  large,  the  tabulation  problem  is  at  the  moment  the 

most  difficult.  Suppose  that  we  allow  each  x^  to  assume  one 

hundred  values,  say  x^^  -  0,  4  ,  ...,  9?A  •  Thien  the  total 

number  of  grid-points  required  to  tabulate  f|^(  ,  x^  ,  .  .  .  ,  x^^ ) 

PN 

will  be  10  .  If  N  •  1,  this  is  100,  a  trivial  number; 

if  N  -  2,  this  is  10,000,  a  respectable  number;  and  if 
N  ■  3,  this  l3  1,000,000,  an  impossible  number  at  the  present 


time  . 
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It  follows  that  the  foregoing  method  In  Ita  atralg^  tfor'- 
ward  form  cannot  ^e  used  to  handle  p’^ot  lemo  of  this  nature 

unless  N  or  M  <  2. 

Two  facts  save  this  from  being  an  academic  exercise.  In 
the  first  place,  there  are  a  numt er  of  Important  situations  In 
which  N  or  M  Is  one,  two  or  three.  In  the  second  place,  as 
we  shall  see  below,  there  are  a  number  of  devices  we  can  com¬ 
bine  with  the  functional  equation  technique  In  order  to  treat 
higher  dimensional  proMems. 

These  are 

(l)  (a)  Lagrange  multipliers, 

(b)  Functional  approximation, 

(c)  Successive  approximations. 

We  shall  discuss  tbiese  Ideas  In  turn. 

1^1,  Reduction  by  One  Variable 

In  view  of  the  tremendous  dlfferenc''  between  the  memory 
rtqui)7eiaents  for  functions  of  two  variables  and  functions  of 
three  variables.  It  Is  of  Interest  to  point  out  tbiat  trans¬ 
portation  processes  Involving  N  sources  can  he  treated  t  y 
means  of  functions  of  N  -  1  variables.  Hence,  problems 

9 

Involving  two  sources  are  easily  resolved,  while  problems 
Involving  three  sources  can  be  treated  wltb  the  best  of  current 

■aohlnes . 

To  obtain  this  reduction  In  dimensionality,  we  observe 
that  SB  yet  we  have  made  no  use  of  tbie  fact  that  supply  Is 

•qiMil  to  demand. 
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(1) 


N 

2 

1-1 


X,  - 


M 

2y 

j-i 


j* 


Prom  t^‘ls  It  follows  that  th»*  values  of  x^,  x^, 
determine  the  ’'alue  of  Xjg ,  onre  we  have  specified  the 
Hence 


(2) 


.x^) 


» X  2 


In  much  of  analysln,  dimensionality  plays  an  Inessential 
role.  In  computational  work,  it  Is  a  ^ aalc  consideration. 

1-.  [.agrange  Multipliers  and  Dynamic  Prograinmlng * 

We  have  anoth.er  very  powerful  way  of  evading  the  curse  of 
dimensionality.  Returning  to  the  allocation  prol lem  discussed 
initially,  consider  the  proV lem  of  maximizing 

N 

^  ^  ^  P(x^  .x^ . x^)  -  2  ^ 


subject 

to  th.e  constraints 

N 

(2)  (a) 

2  X,  <  X, 

1  =  1  ^ 

N 

(b) 

(c) 

X 

IV 

o 

Observe  tJ  at  we  iave  kept  one  ''onstralnt  an  equality,  one  an 


First  presented  In  [jj  . 
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Inequallty.  Again  there  are  some  reasona  of  convenlen'e, 

Aa  we  know,  thla  problem  can  be  treated  ty  means  of 
functions  of  two  varlatles.  However,  there  Is  a  great  Incen¬ 
tive  for  reducing  the  problem  to  one  that  can  \e  handled  by 

functions  of  one  variable. 

What  we  do  Is  combine  the  functional  equation  technique 
with  the  classical  Lagrange  multiplier  formalism.  Consider  the 
problem  of  maximizing  the  new  function 

N  N 

(3)  0(x.,Xp,,.,,  X.. )  “ 

1_1  1  1  1-1  ^ 

subject  to  the  constraints 

N 

(^)  (*)  2  <  ^. 

1-1  ^  “ 

(b)  >  0, 

where  X  Is  an  as  yet  undetermined  parameter. 

For  fixed  X,  lntrodi:oe  the  sequence  of  functions 

N  N  " 

(5)  '■nh)  -  . 

whwre  Rj^  Is  defined  only  by  (4).  Then,  as  before,  we  readily 
eonpute  the  sequence  f^^^ns  of  the  relations 

(6)  f|^(x)"  Ma  X  I  g  ^  ^  ^ 

0<Xj^x 

.’.et  x^(X),  Xj^(x)  re  a  set  of  values  yielding 
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t!.e  maximum  of  0(  x ^  ,  .  .  .  )  .  Then  we  assert  that  these 

values  yield  the  solution  to  the  problem  of  maximi¬ 
zing  (l)  subject  to  the  constraint  In  (2)  where  y  Is 
determined  by 

N 

C')  y  -  2  »,)<,(’''• 

1-1 

To  prove  tfls,  pro'eed  hiy  con t radl '' t Ion  .  Suppose  that 

th  ^»re  existed  values  (  z,  ,  z,, ,  .  .  .  ,  z., )  satisfying  (2)  such  that 

i  c  N 

(^)  F(z^  .z^,  .  .  .  ^  F(  (  A  )  ,  x^C  X  )  ,  .  .  .  ,Xj^(  X  )  )  . 


Then 

N 

(■<)  F(z^,Z2....,Zj^j^  —  X  ^  ^  t  2  ’  ’  ’  ‘  ~ 


F(x^(  X)  .XpC  X) . X) )  -  Xy 

N 

-  P{x^(x),X2(x),...,Xyj(x))  -  X2ajXj(x). 

This,  however,  yields  a  contradiction ,  since  the  Xj^(x)  were 
ol  talned  as  a  solution  to  ('),  sulject  to  the  constraints  of 
(4). 

Although  th.ere  Is  no  difficulty  in  letting  tt'.e  results 

justify  trie  netfoi  In  any  particular  application,  there  are  a 

numler  of  Im.portant  facts  which  remain  to  te  verified.  We 

suspect  t^at  as  X  varies  from  -  oo  to  oo  that  the  value 
N 

of  will  .'ary  between  Its  maximum  and  minimum,  and, 

1-1  ^ 

fui’tt.ermore  that  this  variation  will  he  monotone  and  continuous. 
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The  monotonlclty  la  not  only  of  theoretical  Importarce, 

but  of  practical  significance  In  deter'mlnlng  the  value  of  X 

N 

for  which  ■  y;  Oi’oss  and  Johnson,  .  Some 

applications  of  ti  ls  technique  will  he  found  In  ,  [l^  . 

16.  PlecuBBlon 

TTie  Importance  of  tie  procedure  outlined  at  ove  resides  In 
the  fact  tlat  It  enables  us  to  partition  a  problem  originally 
requiring  a  sequence  of  functions  of  two  varla^le3  Into  a  se¬ 
quence  of  problems  requiring  funcclons  of  one  varlalle. 

Th.ere  Is  no  difficulty  In  extending  these  techniques  to 
treat  the  case  wh,ere  there  are  M  constraints.  What  we  gain 
In  reducing  dimensionality  on  one  hand,  we  must  pay  for  In 
■oiltl-dlmenslonal  search  on  the  other. 

As  we  know,  the  Introduction  of  Lagrange  multipliers  Is 
equivalent  to  Introducing  dual  variables;  cf.  Kuhn  and  Tucker, 
What  we  have  done  above  Is  to  operate  partially  In  the 
original  space  and  partially  In  thie  dual  space;  partly  In  th.e 
■pace  of  "resources”  and  partly  In  thie  space  of  "prices." 

17.  l\inctlonaI  Approximation 

In  the  previous  sections,  when  we  h.ave  dls'^ussed  the 
computational  solution  of  functional  equations,  we  h.ave  ta  Itly 
•quated  the  concept  of  a  function  f(>)  defined  over  an  Inter'- 
val  with  a  set  of  values  <f(k6)  ,  where  UL  ■=  2,  anJ 

A  la  aorae  grid  size.  Th',e  finer  the  grid,  the  more  values  th.at 
■UBt  be  computed.  Similarly,  a  function  of  two  varlalles, 
f(x,y),  Is  equivalent  to  a  g#»quen 'e  of  values  . 
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As  wo  Increase  tlie  nur.t  er  of  Independent  varlalles,  the 
number  of  grid— points  goes  up  at  an  exponential  rate.  It  Is 
thils  fact  that  defeats  thie  effective  use  of  the  algorlth.ms 
presented  above  In  a  number  of  significant  processes. 

It  follows  that  one  way  to  defeat  this  exponential  growtbi 
In  the  Information  required  to  sptclfy  a  function  Is  to  use  a 
different  description. 

Consider,  for  example,  a  power  series  expansion 

(1)  f(x)  .  2  ax", 

n«0 

convergent  for  0  <  x  <  a.  If  we  truncate  the  series  and  use 

ir  - 

thiC  polynomial  ^  approximation  to  the  function, 

n«0 

we  see  th.at  r(x)  Is  determined  for  all  x  In  [0,a']  by  the 
N  1  coefficients,  a^,  1  -  0,  1,  ...,  N,  and  thius  by 
(N-fl  )  quantities . 

Power  sei  cs  expansions  have  the  drawback  of  being 
associated  with,  analytlolty  and,  In  addition,  of  not  providing 
uniformly  good  approximation  over  the  entire  Interval.  Let  us 
then  use  Instead  an  orth^.onormal  expansion 

00 

(2)  f(x)  --  2 

n-0 

where  the  functions  are  elements  of  a  complete  oi'th,o— 

normal  system.  For  a  finite  Interval,  two  particularly 
Important  choices  are  those  of  trigonometric  functions, 

/sin  kx,  cos  kx! ,  and  of  Legendre  polynomials. 


We  write 


(5) 
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N 


-  2 


n-0 


Where 

(4)  a  -  r(x)0  (x)dx. 

"  0  ^ 

In  evaluating  this  Integral,  we  don't  wish  to  use  a 

Plemann  sum,  say 

M 

(5)  ^  2  r(kA)0  (kL)L. 

"  k-0 

since  this  will  Involve  tt- e  calculation  of  f(k/^  for  all  k, 
precisely  the  type  of  computation  we  wished  to  avoid. 

Consequently,  we  <*mploy  a  numerical  Integration  formula  of 

the  form 

.  M 

(6)  g(x)dx  -  2 

where  are  fixed  points  Ir,  L'^,aJ  ,  Independent  of  g(x) 

but  dependent  on  M,  and  the  are  likewise  fixed  coeffi¬ 

cients  Independent  of  g(x),  hut  d<  {)endent  on  M. 

Thus 

1-1  1-1 

■ince  the  quantities  calculated  oncp  and  for 

all. 

Observe  the  Inter’estlng  fact  al  out  this  formula  that  tfio 
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value  of  ,  and  thus  of  r(x)  Is  made  to  depend  upon  the 
values  of  f(xj^)  at  a  fixed  set  of  points  . 

Let  us  see  then  h.ow  the  calculation  proceeds.  Turning  to 
tlie  recurrence  relation 


(8) 


^  ‘‘n-i ( *-*11  >] 


suppose  that  x  la  restricted  to  an  Interval  .  Starting 

with  the  function  f^(x)  -  g^(x),  k_  reduce  fj^(x)  to  a 
sequence  of  coefficients 


We  see  from  the  above  discussion  that  to  determine  f^(x),  we 
need  only  calculate  the  values  .  Hence,  we  compute 

these  from  the  relations 


(10) 


-  Max 


0< X <x  ^ 


gj^Cx^) 


fl(Xi-X^) 


The  values  of  f^(xj-X2)  are  determined  from  (7). 

Having  computed  f2(>’.^),  i  ••  1 ,  2 ,  .  .  .  ,  M ,  In  tbils 
fashion,  we  determine  tb.e  new  coefficients  a|  ^  using  (7). 
The  function  thus  reduced  to  a  sequence  of  coeffi¬ 

cients 


(11) 


We  now  repeat  this  process  to  determine  as  many  elements 
of  tf e  sequence 


as  desired. 
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A  numler  of  questlonc  remain  before  It. la  te  :.Mnlque  can  te 
applied.  We  muet  determine  N  In  (3)  and  M  In  (7),  and  the 
type  of  orthonormal  sequence.  Tlie  choice  of  N  and  M  depend 
upon  the  accuracy  desired  and  the  facilities  available. 

Both  the  trigonometric  functions,  isln  Nx ,  cos  Nx » ,  and 

> 

the  Legendre  jolynomlals  possess  simple  recurrence  relations 
which  permit  the  N— th  member  of  the  sequence  to  be  computed 
from  the  values  of  the  first  members. 

As  far  as  quadratic  formulas  are  concerned,  It  Is  prot ably 
best  to  use  Oausslan  quadrature,  wtilcr. ,  as  we  know.  Is  exact 
for  polynomials  up  to  degree  2M  -  1  If  M  points  are  used  In 
(b).  For  an  application  of  this  tec..nlQue,  see  . 

18.  Cebycev  Approximation 
The  approximation 

(1)  f(x)  - 

n«u 

la  equivalent  to  chooalng  the  coefficient  according  to 

■aaiv— square  approximation.  If  the  b^  are  determined  so  as  to 
KlniBlze  the  mean-square  deviation 

(2)  rf(x)  - 

0  L  n-0 

we  find  that  b^  “  • 

However,  mean-square  deviation  Is  less  desirable  th.an 
Cebycev  approximation, 

N 

hln  Max  |f(x)-  ^b0(x)'. 

bi  0<x<a  n-0 


(5) 


p-i2b^ 

-28- 

Unfortunately ,  no  slmpl**  repreaentatlon  for  the  minimizing 
corresponding  to  (17,4)  exists.  Nevertheless,  there  are 
available  feasll le  computational  techniques  for  determining  the 
minimizing  In  (3). 


19 «  Functions  of  Several  Variables 

In  the  previous  sections,  we  have  shown  how  a  function 
defined  over  G^.a^'  may  he  described  by  a  relatively  small  set 
of  parameters.  The  same  process  can  le  applied  to  a  function 
of  two  variables,  f(x,y),  defined  over  0  <  x,y  <  a, 


(1) 


f  ( X ,  y ) 


N 


m  ,n-0 


a  ^  ( 

mn  m 


x)^^(y) . 


We  see  that  functions  of  two  variables  will  require 

( N+1 )  (N+2  )/2  coefficients,  wl.lle  those  of  three  variables  will 

3 

require  roughly  N  /*>  coefficients. 

Take  N  -  10,  we  have  functions  of  two  variables  deter¬ 
mined  by  hf,  quantities,  and  functions  of  three  variables 

determined  hy  approximately  200  quantities.  These  numbers 

4  (? 

compare  very  favorably  with  10  and  10  arising  from 
10^  X  10^  grids  and  10^  k  10^  k  10^  grids. 

In  any  particular  problem,  a  certain  amount  of  experi¬ 
mentation  will  be  required. 

Again  an  Important  point  to  stress  Is  that  these  technique: 
allow  us  to  treat  problems  which  cannot  be  treated  by  straight¬ 
forward  tabulation  of  functional  values  at  grid— points. 
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20.  SuccesBive  Approximations 

Let  us  now  dlscues  an  approach  of  entirely  different  type 
to  the  problem  of  solving  multi— dimensional  problems  In  terms 
of  functions  of  a  small  number  of  variables.  We  wish  to  employ 
the  classical  method  of  successive  approximations . 

To  Illustrate  the  workings  of  the  method,  let  us  give  two 
examples  of  Its  use,  one  In  connection  with  the  allocation 
problem  described  above,  and  one  In  connection  with  the 
Hitchcock— Koopmajis  problem. 

Consider  the  problem  of  maximizing 

N 

(1)  h(x^,.,.,x^,y^,...,3r|^)  “  ^  ^Ic  *  ^Ic  ^ 

k*l 

subject  to  the  constraints 
N 

(2)  (a)  ^ 

(b)  -  y,  y^  >  o. 

As  we  know,  this  problem  can  te  treated  ly  means  of  sequences 
of  functions  of  two  varlatles,  ,  and  iy  means  of  3equen:es  of 
functions  of  one  variable  using  Lagrange  multiplier’  techniques. 

Let  us  now  treat  It  y  means  of  successive  approximations. 

Let  . Initial  guess  ^on:ernlng 

the  choice  of  the  y^^  and  consider  the  problem  of  maximizing 


the  function 


(3) 
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h  ( X. 


x^iyi 


(0) 


»y 


(0) 

N 


-  2 

k-1 


subject  to  tb-e  constraint  of  (2a).  This  problem  can,  as  we 
know,  be  resolved  via  functions  of  one  variable. 

Call  a  maximizing  set  of  x^  consider  the 

problem  of  maximizing 


(^) 


h(x; 


(0) 


.  .  ,x 


(0) 

N 


N 


;y 


1 ' 


^  y  ) 

k'  K  '■^k^' 

k-1 


subject  to  the  constralnta  of  (2b).  This  again  Is  a  one— dimen¬ 
sional  problem  in  our  terms.  Call  a  maximizing  set  • 

Tne  pattern  of  procedure  Is  now  set.  We  obtain  alternately 

(  (1)1  I  ( 1  ^ ' 

maximizing  sequences  |  and  j  x^^  ,  with  corresponding 

approximations  to  tfie  desired  maximum  value. 


(O 


(1) 

•u>'m  . ;y 


N  ■'! 


yd)) 

» •  •  • » )  > 


V-  (  ^  ^  ^  y  '  '  •  V 


(l). Jl>l) 

N  ’^1 


,y 


(l-bl) 

N 


We  have  thus  once  again  reduced  a  problem  originally 
repulrV'ig  fuhctlons  of  two  variables  to  one  requiring  sequences 
of  functions  of  one  variable. 


21.  Mono  ton  1‘  Ity  of  pt^o  xl  mat  Ion 

Let 


Up^^^(x,y)  -  Max  h(x,y^^^),  1  -  0,  1,  ..., 


u, 


(x,y)  -  Max  ti(x^^\y).  1  -  1,  2,  ... 

y 


(1) 


It  l3  clear  that 


?_24-3r 
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(2)  u^(x,y)  <  u^(x  ,y  )  <  •  •  •  . 

Hence,  the  sequence  |u^(x,y)|  convergeB  monotonlcal ly .  It 

'  j 

Is,  however,  not  clear  tf at  It  converRea  to  the  al aolute  maxi— 
■turn.  This  requires  a  separate  discussion  which  we  shall 
present  elsewhere. 

22,  Approximation  In  Policy  Space 

This  monotonlc 1 ty  of  approximation  Is  not  ac'ldentel.  o 
type  of  approximation  we  have  been  employing  la  a  particular 
type  of  approximation  in  "policy  space,"  wl  1  '  necessarily 

yields  monotonlclty . 

In  place  of  approximating  to  the  return  functions,  t(  e 
fj^(x,y)  as  defined  in  ,  the  usual  method  of  successive 
approximation,  we  operate  partially  In  the  space  of  policy 

functions . 

For  a  further  discussion  of  aj:  proxlmatlon  In  policy  space, 
see  Ql]  ,  Chapter  and  for  some  further  applications  of 
successive  approximations  ,  see  '  '♦!  , 

23*  Application  to  Hitchcock— Koopmans  Transportation  Proi  ler. 

One  way  of  applying  these  Ideas  to  either  the  linear’  or 
nonlinear  transportation  Is  the  following.  Let  tr e  shipments 
from  the  3“rd  to  N— th  source  le  assigned  a  rr  1 1  rai'l  ]  y  ,  and 
consider  the  problem  of  detennlnlng  the  slilpments  from  th.e 
first  two  sources  which  will  rrlnlmlze  the  cost  of  supplying  tr  e 
remaining  demand. 
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■nils,  as  we  know,  can  be  done  using  functions  of  one 
variable.  Having  obtained  the  optimal  shipments  from  the  first 
and  second  sources,  we  use  this  shipping  policy  from  the  first 
source,  retain  the  shipments  previously  used  from  the  fourth 
to  N-th  source,  and  consider  the  new  problem  of  determining  the 
shipments  from  the  second  and  third  sources  which  will  minimise 
the  cost  of  supplying  the  remaining  demand. 

Continuing  In  this  way.  It  Is  clear  that  we  obtain  a 
sequence  of  costs  which  decrease  monotonlcally  and  thus  con¬ 
verge.  Again  It  Is  necessary  to  determine  when  there  Is 
convergence  to  the  actual  minimum. 

The  process  can  be  speeded  up,  by  considering  three 
sources  at  a  time  and  one  Lagrange  multiplier  to  retain  the  use 
of  functions  of  one  variable. 

24.  The  Harris  Transportation  Problem 

Consider  the  following  network  which  can  be  used  to 
describe  certain  types  of  flow  of  Information  or  flow  of 
coomodltles . 
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Ab  the  arrows  Indicate,  the  only  permissible  flows  are 
*1  ‘1-fl'  ‘1 


from  to  to  to  Pj^^^ ,  aiid  to 

^1+1* 


We  Introduce  quantities,  which  we  can  call  capacities , 
defined  as  follows: 


(1) 


a.  -  niaxlmum  rate  of  flow  over  the  link  between  P, 


and  P 


l-fl  ’ 


b.  - 


- 


d,  - 


the  same  quantity  for  Pj^  and 
the  same  quantity  for  Qj^  and 
the  same  quantity  for  and 


Assuming  that  we  are  given  fixed  rates  of  flow,  x  and 
y.  Into  and  we  wish  to  maximize  a  prescribed 

function  of  the  rate  of  flow  Into  Pj^  and  At  each  of 

tho  tersilnals,  we  have  a  choice  of  dividing  the  Input  flows 

Into  two  output  flows. 

To  treat  this  problem,  we  Introduce  the  sequence  of 

functions, 

(2)  “  the  maxlmvim  of  the  prescribed  function  of 

the  flow  Into  Pj^  and  given  rates 

of  flow  X  and  y  Into  P^  and 
respectively , 

for  1-1,  2,  ...,N-1,  x,y>0. 

TT)en,  as  above,  we  obtain  the  recuri’ence  relations 

-  Max  f4^i(x^**-x^,y^+y  ), 


(5) 


P-1284 

2-24-58 

-54- 


where  Is  determined  by  the  constraints 

(4)  (a)  Vi  <  ^2  ^2  - 

(b)  0  <  <  a^ ,  ^  <  ^2  -  ^1' 

(c)  0  <  y^  <  c^,  0  <  y2  < 


->  *1  ^2 


->  "*■  ^2 


2^ .  General  Network  Problems 

Networks  of  general  type,  where  the  points  are  Irregularly 
arranged  with  great  freedom  of  Interconinunlcatlon  cannot  be 
treated  In  the  foregoing  fashion.  The  problem  of  determining 
optimal  flow  over  systems  of  this  type  was  first  posed  by 
Harris,  [^  ,  and  has  given  rise  to  a  good  deal  of  research. 

The  general  problem  has  teen  most  successfully  attacked  by 
Ford  and  Fulkerson,  who  have  developed  techniques  for  the 

treatment  of  this  specific  process  which  have  proved  to  have 
much  wider  applicability. 

On  the  other  hand,  stimulated  by  the  same  process, 
Boldyreff,  53,  has  developed  a  very  Interesting  and  flexible 
relaxation  technique,  the  "flooding  technique,"  which  also  has 
a  wide  rsmge  of  applications. 


2-? 

-y~ 


26.  A  Routing  Problem 

Let  us  now  consider  the  following  rroilerr.,  SXjppoae  that 
we  have  a  set  of  N  points  In  the  plane  or  In  space,  with  t:  e 
property  that  every  two  points,  P.  and  P,,  have  an  associated 
number,  '•fhlch  we  can  call  the  tl.Te  required  to  travel 

from  to  Pj .  It  Is  not  necessary  to  assmre  that  -  d^^. 

To  take  account  of  the  fact  tt.at  In  any  [articular  situation 


two  points  may  not  be  mutually  accessl!  le,  we  can  let  d^^ 


00 


Olven 


the  matrix  (d^^),  wliere  d^^^  -  0,  the  [  rol  lem  Is 


to  trace  a  route  of  minimum  time  from  to  . 

1  N 

To  treat  this,  Introdu-'e  the  N  —  1  quantities,  , 

defined  by 


(1)  ■  the  minimum  time  to  travel  from  to  Pj^ . 


It  is  easy  to  see  that 

(2)  r  -  Min  [d  +  f  ]  . 

1  J/1  J 

Since  this  system  of  non— linear  equations  does  not  deter- 
■Ine  the  sequence  recurrently,  we  must  use  some  met.hod 

of  successive  approximations  to  obtain  tie  f  ^  . 

Perhaps  the  simplest  Is  one  hased  upon  approximation  In 

policy  space.  Let 

(3)  fj  '  -  1.1,  2,  -  N  -  1 , 


and 
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1  J^l  L  Ij  J  i 
for  k  -  1 ,  2 ,  ...  . 

Since  this  process  corresponds  to  exaunlnlng  first  all 
direct  routes  from  1  to  N,  then  all  that  make  one  step,  aind 
so  on,  it  Is  clear  that  we  will  have  monotone  decreasing  con¬ 
vergence.  Further  discussion  concerning  the  equations  will  be 
found  In  [5]  . 

27.  The  Traveling  Salesman  Problem 

As  an  exa.mple  of  a  problem  of  closely  related  type  where 
the  direct  functional  equation  technique  falls,  consider  the 
problem  of  drawing  a  polygonal  path  of  minimum  length  through 
N  given  points  In  the  plane,  P^,  P^,  ...,  Pj^. 

It  Is  clear  that  the  principle  of  optimality  Is  still 
valid.  No  matter  what  part  of  the  path  has  already  been  tra¬ 
versed,  the  remainder  of  the  path  must  be  of  mlnlmxim  length. 

What  prevents  a  simple  application  of  recurrence  relations  Is 
the  fact  that  we  must  keep  track  of  where  we  have  been.  The 
problem  thus  has  certain  features  In  common  with  a  variety  of 
"excluded  volume"  problems  arising  In  mathematical  physics. 

Furthermore,  it  Is  a  very  nice  example  of  the  virtual 
Impossibility  of  gauging  the  level  of  difficulty  of  a  simply 
stated  combinatorial  problem.  Recently,  linear  programming 
techniques  plus  Ingenuity  have  proved  successful  In  solving 
particular  questions  of  this  nature,  see  Dantzlg  and  Johnson,  [58. 

To  treat  this  problem  by  means  of  fiinctlonal  equations  and 
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successive  approximations,  let  us  tegln  with  the  question  o^ 
tracing  a  path  through  N  given  points  wnlch  must  start  at 
and  end  at  P^. 

Introduce  to  this  end  the  sequence  of  functions 


(1) 


rCQj.p. 


,Q^)  -  the  minimum  length  of  the  remaining 
path  from  to  Pj^,  having  teen 

through  Q^,  ^2'  *  *  •  '  * 


Here  the  are  particular  elements  of  the  Pj— set. 

It  is  easy  to  see  that  we  obtain  the  following  relations: 


(2) 


f(Pj)  .  Min 


and  80  on. 

It  remains  to  discuss  the  computational  feasibility  of  a 
solution  based  upon  this  algorithm.  The  tabulation  of  f(P^,Pj) 
requires  (N-l)  values;  that  of  f(P,Pj,P^^)  requires 
(N-l)(N-2)/2.  What  Is  of  great  help  Is  the  fact  that  the  order 
of  In  (1)  Is  of  no  Impor^-ance ,  so  that  the  tabulation  of 

f (Qi,Q2, . . . ,0^)  requires  (N-i ) (N-2 ) • • • (N-k  )  A  !  entries, 
rather  than  (N— 1 ) (N— 2) • • • (N— k ) . 

The  maximum  number  of  entries  will  be  required  when 
k  •  [H/2]  .  To  Illustrate  the  order  of  magnitude  of  these 
quantities,  we  have 

(5)  lOC^  -  252,  l6Cg  -  12,970. 
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It  follows  that  with  current  machines  It  would  he  possible 
to  solve  problems  of  this  type  In  a  direct  fashion  for  N  <  17. 

Let  us  note  In  passing  that  the  saune  technique  can  be 
applied  to  problems  of  optimal  trajectory’  arising  In  rocket 
problems,  and  In  the  study  of  general  variational  processes; 
cf.  Cai  talno  and  Dreyfus ,  [25]  ,  and  [6]  ,  [l]  . 

23.  Successive  Approximations 

To  treat  problems  of  larger  magnitude,  we  cam  combine 
functional  equation  techniques  with  the  tnethod  of  successive 
approximations . 

Let  ^  proposed  route.  To  test  this,  let 

us  examine  the  sub-route  and  pose  the  problem  of 

pursuing  a  path  from  P^  to  Q^,  going  through  the  points 
Q2  » •  •  •  minimal  length. 

This  problem  can  easily  be  resolved  computationally,  using 
the  recurrence  relation  method  of  the  preceding  section.  Let 
^1^2^3’'’^  be  the  new  path  of  minimum  length  through  these 
ten  points,  and  let  us  test  In  this  way  the  set  of  points 
R^R^R.^RgQgPj^QPj^^Pj^2Pi5Pl4*  Continuing  In  this  way  we  obtain  a 
monotone  sequence  of  decreasing  lengths.  Again  It  will  be 
necessary  to  study  whether  or  not  this  scheme  of  successive 
approximations  converges  to  the  absolute  minimum. 

29.  A  Class  of  Scheduling  Problems — The  Book— binding  Problem 

Let  us  now  turn  to  another  class  of  problems  which  require 
maximization  over  permutation. 
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Suppose  that  we  have  N  n^anuscrlpts  which  must  oe  printed 
and  bound.  In  that  order,  before  being  published.  Suppose 
further  that  we  have  one  printing  presr  amd  one  binding  machine. 
Olven  the  qiiantltles 

(l)  (a)  -  the  time  required  to  print  the  1-th  book, 

(b)  -  the  time  required  to  bind  the  1-th  book, 

the  problem  Is  to  detemlne  the  order  In  which  the  N  manu¬ 
scripts  should  be  processed  so  as  to  minimize  the  total  time 
required  for  their  printing  and  binding. 

•Ms  problem  Is  representative  of  a  large  class  of 
questions  of  this  nature  which  arise  In  scheduling  theory.  A 
very  simple  solution  of  this  problem  ►'as  given  by  Johnson,  3^2  » 
and  a  derivation  of  this  solution  by  functional  equation  tech¬ 
niques  was  given  In  [7^  . 

If  we  add  a  third  operation,  say  typing,  the  problem 
appears  to  enter  the  hopeless  case.  At  the  present  time,  we  do 
not  even  possess  amy  reasonable  approximate  pol.LCles. 

30.  The  caterer  Problem 

It  Is  possible.  In  a  number  of  cases,  to  reduce  scheduling 
problems  to  maximization  problems  of  the  type  encoxontered  In 
the  Hitchcock— Koopmans  transportation  problem.  Having  done 
thla,  we  can  Introduce  functional  equations  by  various  artifices, 
tee  exaoqple  of  this  Is  In  connection  with  "warehousing"  problems, 
of.  00,  Dreyfus,  (^  ,  and,  also,  [ji]  .  Here,  we  shall  discuss 
another  example,  the  "caterer"  problem. 
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Let  us  state  the  version  of  this  problem  given  by  Jacobs, 

,  or  Prager,  . 

"A  caterer  knows  that  In  connection  with  the  meals  he  has 
arranged  to  serve  during  the  next  n  days,  he  will  need  r 

J 

fresh  napkins  on  the  J-th  day,  for  J  -  1,  2,  ...,  n.  There 
are  two  types  of  laundry  service  available.  One  type  requires 
p  days  aiid  costs  b  cents  per  napkin;  a  faster  service 
requires  q  days,  q  <  p,  but  costs  c  cents  per  napkin, 
c  >  b.  Beginning  with  no  usable  napkins  on  hand  or  In  the 
laundry,  the  caterer  meets  the  demands  by  purchasing  napkins  at 
a  cents  per  napkin.  How  does  the  caterer  purchase  and  launder 
napkins  so  as  to  minimize  the  total  cost  for  n  days?" 

IVils  problem  can  be  resolved  by  linear  programming  tech¬ 
niques  In  some  cases,  see  the  above  references  amd  also  Oaddum, 
Hoffman,  and  Sokolowsky,  . 

What  Is  Interesting  about  this  problem  from  the  standpoint 
of  dynamic  programming  and  functional  equations  Is  that  It 
appears  upon  first  glance  to  be  a  problem  requiring  functions  of 
q  variables.  As  It  turns  out,  however,  the  linearity  of  the 
process  permits  us  to  make  a  certain  type  of  preliminary  trams- 
formation  which  reveals  the  true  dimensionality  of  the  problem. 
Surprisingly,  this  Is  p  _  q.  The  same  type  of  transformation 
has  proved  of  great  service  In  connection  with  a  number  of 
variational  problems  arising  In  the  theory  of  control  processes 
and  elsewhere,  see  ,  [j§  . 

In  the  present  case,  these  transformations  reduce  the 
problem  to  that  of  determining  the  maximum  of  the  linear  form 
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(1) 


L(v)  -  V,  •••  -fv 

12  r 


subject  to  the  constraints 


(2)  (a)  >  0, 


(b)  , 

Vi  <  b^, 


V,  <  t,  , 

k  —  V. 


V  4-v  -f‘**4-v  <rl- 


V  ,  + 

n— k+1 


+  V  <  b 
n  -  n 


A  further  surprising  fact  about  this  problem  1?  tt.at  we 
can  exhibit  an  explicit  analytic  solution,  a  property  ttiat  In 
quite  rare  In  this  domain. 

31.  Bottlene  k  Prob'^ems 

Let  us,  wlt.bout  going  Into  any  detail,  mention  a  class  of 
problems  which  wc  have  called  "bottleneck"  problems  because  the 
operation  of  t.r.e  system  depends  at  eao:.  stage  upon  tb.e  scarcest 

reeoui’ces . 

The  gene  ’al  question  Is  ti.at  utilizing  a  complex  of  Inter— 
dependant  Industries  to  produce  one  or  two  essential  Items. 
Using  a  "lumped"  model  of  economic  Interaction  we  '"onslder  tl  e 
state  of  the  system  at  any  time  to  be  specified  by  a  capacity 
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vector  x(t)  and  a  stockpile  vector*  y(t). 

Considering  first  a  continuous  process,  since  tlieae  are 
usually  rr.ore  amenable  to  solution,  we  meet  the  problem  of  deter'— 
mining  vectors  i:(t)  and  w(t)  so  as  to  maximize  the  Inner 
product 

(1)  I(x,y)  -  (x(T),a)  (y(T),b), 

where  x  and  y  are  deterTr.lned  by  linear  equations 
-  AjX  -f  n,y  +  CjZ  +  DjW,  x(0)  -  c, 

^  .  A.  .  B^y  f  CjZ  +  Djw,  y(0)  -  d, 

under  appropriate  proportionality  assumptions,  and  the  vectors 
z  and  w  are  subject  to  further  feasibility  constraints 

( 3 )  Ez  +  Pvf  <  0  Hx  -f  ly 

for  0  <  t  <  T. 

The  novel  features  of  the  problem  are  introduced  by  the 
combination  of  linear  equations  and  linear  constraints.  The 
continuous  version  can  be  resolved  explicitly  In  a  number  of 

cases,  see  [l]  ,  [jt  •  ^  addition,  Lehman  has  devised  a  con¬ 

tinuous  version  of  the  simplex  technique  wh.lch  seems  quite 
promlnlng,  . 

The  discrete  version  can  be  simply  treated  I y  computational 
techniques  If  the  number  of  state  variables  does  not  exceed  3. 

A  transfonnatlon  of  the  proclem  enables  us  to  reduce  t.he  numb  er 
of  varlab'les  by  one,  and  simultaneously  to  keep  all  variables 
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wlthln  a  uniformly  bounded  region;  cf.  Ji j  ,  . 

If  the  constraints  In  (3)  are  of  the  simpler  form 

(4)  Ez  +  PVf  <  0, 

the  techniques  sketched  In  can  be  used  to  reduce  drastically 
the  dimensionality  of  the  problem. 

A  detailed  discussion  of  problems  of  this  type,  together 
with  solutions  of  some  special  cases  Is  given  In  _i] .  Some 
further  results  are  found  In  Lehman,  . 

32.  Slightly  Intertwined  Matrices 

The  fxanctlonal  equation  technique  Is  designed  to  take 
advantage  of  structural  features  of  the  process.  In  the  pre¬ 
ceding  discussion  we  have  utilized  the  multistage  aspects  In  a 
very  natural  way.  Let  us  now  indicate  another  way  In  which 
functional  equations  can  be  employed.  These  results  were 
praaented  in  [32]. 

The  general  linear  programming  problem  Is  that  of  maximi¬ 
sing  a  linear  form  L(x)  -  (x,a)  subject  to  linear  constraints 

of  the  form  Ax  <  b. 

If  A  is  block— diagonal , 


the  problem  breaks  up  Into  a  set  of  r  Independent  problems  of 
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amaller  dimension.  Once  again  we  are  concerned  with  dimension¬ 
ality  in  connection  with  computational  feasibility. 

Similarly,  If  A  Is  block— triangular ,  which  Is  to  say 
that  there  are  zeroes  above  or  below  the  diagonal  matrices, 
the  problem  reduces  to  a  sequence  of  problems  of  lesser 
difficulty. 

Here  we  wish  to  show  how  functional  equation  techniques  can 
be  utilized  to  treat  a  class  of  problems  In  which  A  has 
approximately  a  block-diagonal  form. 

Specifically,  let  us  consider  the  question  of  determining 
the  maximum  of 

3N 

(2)  L^(x)  -  2 

^  1  =  1  ’■ 


^3N-2 , 5N-2^5R-2  ®3N-2,3N-i’^3N-1  ^3N-5,3N^3N  ^3N-2' 
^3N-1 ,3N-2^3N-2  ®3N-1 ,3N-1^3N-1  ^3N-1,3N^5N  ^  '^3N-i» 
‘"3N,3N-2''3N-2  ^3N,3N-l''3N-T  ®3N,3N^3N  =3N' 


It  l8  aBBumed  that  2.  ^  >  0 ,  wit:  eufflclentl 

many  >0  so  that  th »  maximum  Is  not  Infinite. 

Let  U8  now  define  a  sequence  of  fun'^tlons  of  z, 


(5) 


r^(z)  -  Max  L^(x) , 


where  the  are  aul  ject  to  tnc  constraints  given  a^  ove  ,  wit:, 

the  exception  that  the  last  constraint  Is  now 


(6) 


*3N, JN-2'‘5M-2  ®3N,5N-i’'3H-''.  *  ®3N,3N*3N  C  '• 


Qnploylng  the  principle  of  optimality,  we  see  that  the 
sequence  fc^iz)!  satisfies  tt.e  recurrence  r-elatlon 


(7) 


fj^(z)  - 


Y  Y  4-  Y 

-^N-2  '^:5N-1  3N 


■f  r  f  c  — l  y  ) 

'  N-r  3N-3  N-1  ’ 


N  >  1, 


with  the  variables  x 


su'.ject  to  the  constraints 
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(8) 


^3N-2,3N-2’'3N-2  ^3N-2,3N^3N  ^  °3N-2’ 

a5N_i^3N_2''3N-2  ^3N-1,3N-1^3N-1  ^3N-1.3N^3N  ^  ' 


*^3N,3N-2^3N-2  ®3N,3N-1^3N-1  ^3N,3N^3N 

^N-l’^3N-2  -  ^3N-3’ 


‘3N-2/3N-1 ’*3N  ^ 


>  0. 


The  function  ^'q(z)  Is  Identically  zero. 

33.  Reduction  In  Dimensionality 

Let  U8  write  tr.e  recurrence  relation  of  (32.7)  In  the  form 


(1) 


n 


fj^(z)  -  Max  iMax 

^3N-2  lj^3N-l’^3N 


r 


[•••] 


-  Max  ,Max 

^3N-2  L 


■*■  ^n-1^^3N-3“^N-1*3N-5^ 


where  Rj^  Is  the  region  In  space  defined  by 


(2) 


®5N-2,3N-1^3N-1  ®3N-2,3N*3N  -  ^31^2  ~  ^3N-2 , 3N-2^3N-2 ’ 


®3N-1 ,3N-1^3N-1  ■*■  ®3N-1,3N^3N  ^  ‘^3N-1  “  •3N~l  ,3N-2*3N-2 ' 
®3N,3N-1^3M-1  ■*■  ^3N,3N^3N  ^  ^  ~  ^3N,3N-2*3N-2' 

*3N-1’^3N  ^ 


Thus  we  can  write 


f^(z)  -  Max 

Y 

*3N-2 


Sn^^3M-2’*^  ■*“  ^N-l^'^3N-3“*N-1^3N-2^ 


(3) 


where  ^  constrained  by 


(4)  0  <  V 


<  KlnU 


3N-^ 


5N-2 


N-1 


1 


®3W-2,5N-2’  ^3N-1,3N-2  ®3N.3N-2^ 


The  function  gj^(y,z)  1b  readily  determined,  since  the 
maximum  over  Is  attained  at  a  vertex  of  the  region. 

34.  Slightly  Intertwined  Symmetric  ‘latrlces 

Let  us  now  consider  the  problem  of  resolving  a  set  of 
linear  equations  of  the  forms 


®21^1  ®22^2  ^23^5  "  ^^2’ 

®31^1  ^52^2  ®33''5  ^l""^ 

b^x^  a^^x^  t  aj^^x^  a^^x.  -  c^. 

^54^4  ®55*5  *  ®56^6  "  S' 

S4S  ^  ®65S  ^66S  SS 


c 


6’ 


*3N-5 , 5N-2^3N-2  ®3N-2 , 3N-1^3N-1  ®3N-2,3N^3N  "  ^3N-2' 
*3N-l,3^*-2^3N-2  ®31’-1 ,3N-1^3N-1  ■*■  ®3N-1.3N^3N  "  ^3N-1  ’ 
®3N , 3N-2^3N-2  ®5N,3N^3N  “  ^3N' 

A  matrix  of  the  type  appearing  above,  we  shall  call 


sliJthtly  Intertwined.  It  arises  In  a  variety  of  physical, 
anglnaerlng,  amd  economic  problems  Involving  multicomponent 

aystams  with  weak  coupling. 


I/€t  US  Introduce  the  matrices 


p-i:  -4 
2-24-^^  4 
-4fw 


(2) 


for  k  •  1,  2,  ....  N,  and  tb.e  vectors 


(3) 


X  “  (^T).  ^  3^/  ^  "  ('Tl,  T»^1w) 


■3k-2’  3k- 


3k-2  '  3k-l  '  3k 


Since  the  matrix  of  coefflclentB  Is,  by  assumption,  positive 
definite,  the  solution  of  the  linear  system  (l)  Is  equivalent  to 
determining  tlie  minimum  of  the  Inhomogeneous  quadratic  form 


(4) 


(x  ,A^x  )  4  (x  .A^x  )-»-•••  4  (x  ,A^x  ) 


/-./I  1\  r\  /  2  2  \  ^/NN\ 

-  2(c  ,x  )  -  2(c  ,x  )  -  •••  -  2(c  ,x  ) 


+  2b^x^x^  2b^x^  x^  +  ...  ^  2b^,  ,x,..  ,x. 


'N-1''3H-3''3N-2‘ 


For  N-1,2,  ...,  and  -oo<z<oo,  let  us  Introduce 
the  sequence  of  functions  of  the  variable  z  defined  by 


(5) 


f^(z)  .  Mln^ 


I  N 


N 


1  1-1 


2  (xhA.xh  -  2  I  (chxh 


1-1 
N-1 


We  then  rave  tt.e  following  recurrence  relation: 


{>  ) 


rjz)  -  Min 


N 


^^3K’^3N-1 


'  /  N  A  N\ 

I  ( X  ,  Aj^x  )  +  2xz 


5N 


2(:  ,x  )  4  ^N-1  N-1^3N-2^  ’ 


-j 
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Computational  Aspects — I 

Since  tl^e  function  f^(z)  la  readily  determined,  we  -^an 
compute  the  sequence  |r^(i’)|  at  the  exyenae  of  a  minimization 
over  a  >-dlm0n8 Iona  1  region.  Ttils  minimization  may  be  greatly 
speeded  up  upon  using  tl,e  convexity  propertlea  of  the  functions 
Involved.  Although  no  optimal  m.eth  oda  are  knowr  for  multi¬ 
dimensional  problems,  the  one— dimensional  method  presented  In 
be  employed  In  an  Iterative  manner. 

Writing  (3^.'  )  In  the  form 

(1)  ?zx^j^ 

^3N-2  J^3N'*3^^-l  “ 

-  2(  c  ,x  )  '  ^  i  ' 

I 

we  see  that  It  reduces  to 

(2)  rgj^(z,y)  . 

y  ^ 


wti.'re 

(3)  gM(Ziy)  -  Min  i(x^,Ax^)  +  2zx  -  2(c\x^)  , 

X  Y  ^  ^ 


upon  identifying  '^tls  new  relation  1: 

suited  to  the  technique  described  In  . 

The  computation  of  the  functions  |gj^(z,y)|  Is 
of  the  computation  of  thie  sequence  |fj^(z)()  .  Observe 
0OBq>utatlonal  approach  involves  no  divisions. 


new  well- 

independent 
th  at  th  Is 
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36.  Computational  Aspects — II 

Another  approach  to  the  computational  solution  reposes 
upon  the  easily  established  fact  that  ®  quadratic 

In  z  for  each  N,  l.e. 

(1)  fjj(z)  -  u^  4- 

where  Uj^,  Vj^  and  Wj^  ere  Independent  of  z.  This  Is  the 
same  device  used  In  connection  with  Jacobi  matrices,  see  • 
Substituting  In  (3^.6),  we  obtain  the  equation 

(2)  Vj^z  +  w  z^  -  Min  ["(x^.A  x^)  2zx,j. 

^’^3N'*3N-1'^3N-2^  ^ 

-  2(c  ,x  )  +  Uj^^  -h 

2  2 

"*■  ^N-1^3N-2*'N-iJ  • 

Upon  performing  the  minimization  and  determining  the  minimum 
value  of  the  right-hand  side,  we  ootaln  recurrence  relations 
connecting  the  triple  with  the  triple 

^^-1'''n-1’”n-1^* 

This  affords  an  alternative  computational  technique. 

The  problem  of  determining  the  largest  and  smallest  char¬ 
acteristic  values  may  be  approached  In  a  similar  fashion,  cf. 

oi.  &]. 

37.  Reliability  of  Multi— component  Systems 


A  fundamental  problem  In  tie  design  of  electronic  systems, 
switching  networks,  computing  devices  and  automata.  Is  that  of 
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conetruotlng  more  reliable  devices  from  less  reliable  components. 
Essentially,  It  becomes  a  question  of  minimal  duplication.  Some 
general  discussions  of  these  problems  may  \  e  found  In  [ifi  . 

As  a  sample  of  the  type  of  problem  that  cari  be  treated 
using  functional  equation  techniques,  let  us  consider  the 
following. 

Let  us  suppose  that  the  device  we  wish  to  design  will 
consist  of  a  number  of  stages  each  of  which  feeds  Its  successor. 

Thus 


Input 


->  Output. 


The  reliability  of  the  device  will  be  Interpreted  to  be 
the  probability  that  It  operates  successful,',  and  this  In  turn 
will  be  taken  to  be  the  product  of  the  reliabilities  of  the 

Individual  stages. 

In  many  cases,  the  overall  reliability  Is  too  low  for  Its 
Intended  use.  One  way  to  Increase  the  reliability  Is  to  Intro¬ 
duce  a  number  of  duplicate  components  In  parallel  at  various 

stages .  Thus 
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Input 


We  assume  that  we  posseaa  switching  techniques  which  will  auto¬ 
matically  Introduce  a  new  component  Into  the  circuit  at  any 
stage  If  the  first  component  used  Is  faulty.  The  reliability 
of  the  entire  system  can  now  be  Improved  by  duplication  of  com¬ 
ponents  In  this  fashion. 

The  process  cannot  be  carried  to  any  logical  extreme  be¬ 
cause  of  physical  constraints  of  size  and  cost.  Consequently, 
we  have  the  problem  of  determining  optimal  duplication  subject 
to  given  weight  and  size  constraints.  In  addition  to  permit¬ 
ting  first  choice  of  the  number  of  components  at  each  stage,  we 
shall  subsequently  also  allow  a  choice  of  the  type  of  component. 
TTils  latter  feature  Introduces  combinatorial  aspects,  although, 
of  course,  these  are  already  present  In  the  condition  of 
discreteness . 

Assuming  that  at  least  one  component  must  be  used  at  each 
stage,  let 

(l)  probability  of  successful  performance  of 

the  J— th  stage  If  1  -f  Xj  comi.onents  are 
used  at  the  J— th  stage. 


1 
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L«t  the  cost  of  a  component  at  the  J— th  stage  be  Cj  and  the 
weight  be  Wj .  Due  to  the  Increase  In  complexity  of  switching 
circuits  as  Is  increased,  there  Is  no  reason  to  assume 

proportional  cost.  However,  we  shall  do  so  here  to  simplify 
the  notation,  since  the  method  we  present  Is  equally  applicable 
to  the  general  problem. 

The  variational  problem  Is  then  that  of  determining  the 
naximum  of 

(2) 

J-1  ^  ^ 

subject  to  the  constraints 
N 

(3)  (a) 

(t) 

(c )  Xj  -  0,  1 ,  ...  . 

It  is  clear  that  this  problem  may  be  transformed  Into  that 
of  cooputing  a  sequence  of  functions  of  two  variables,  as  out— 
linod  in  and  by  use  of  the  Lagrange  multiplier  technique 

reduced  to  a  problem  involving  a  sequence  of  functions  of  one 
variable.  The  details,  and  the  results  of  some  computations, 
■ay  be  found  in  . 

38.  Different  Types  of  Components 


Let  us  now  suppose  that  at  each  stage  we  have  a  choice  of 
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two  types  of  components,  those  of  an  A— type  and  those  of  a 


B-type.  Let  Cj(A),  Wj(A),  be  respectively  the  costs  and 
weights  for  components  of  the  A— type  at  the  J— th  stage,  and 
Cj(B),  Wj(B)  denote  the  corresponding  quantities  for  B— types. 

Given  the  overall  restrictions  on  weight  and  cost,  we  wish, 
as  above,  to  determine  the  types  of  components,  and  the 
quantities,  which  maximize  the  total  reliability.  We  shall 
suppose  that  the  switching  requirements  are  such  that  at  any 
particular  stage  It  Is  Impossible  to  combine  any  number  of 
A— components  with  any  number  of  B— components . 

Defining  the  sequence  of  fvuictlons,  fj^(c,w).  In  the 
usual  fashion,  with  the  functions  Pj(xj,A),  pj(xj,B)  corres¬ 
ponding  to  obtain  the  relations 


|Max  p^(x^,Ar 


(1) 


f ^ ( c  ,w)  -  Max 


Max  p.(x^,B) 
*1 


where  1  <  <  Min  |^[[c/c^(A)3  ,  G^/w^(aJjJ  in  the  expression 

containing  A,  and  x^  satisfies  the  analogous  constraint 
In  the  expression  containing  B. 

Generally, 


(2) 


fjj(c,w)  -  Max 


^^x  Pjj  ( x^ ,  B )  f ( B )  Xj^ ,  w— Wjj  ( B )  X|^ 


where  Xjj  satisfies  corresponding  constraints. 


(5) 
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^  ^  lC/Cj,(A)j  ,  [w/Wj^(A)]J  , 

1  <  ^  <  Min  ^Lc/Cjj(B)]]  ,  L’^/''w^,(BflJ  , 

In  the  two  maximizations. 

39 «  Sequential  Search 

In  the  remainder  of  the  paper  we  wish  to  discuss  a  number 
of  Interesting  problems  In  which  we  encounter  the  general 
question  of  finding  In  minimum  time  an  element  of  a  finite  set 
possessing  certain  distinguishing  characteristics. 

Considering  the  great  Importance  of  the  problem  and  the 
fascinating  nature  of  the  questions  that  arise,  It  Is  amazing 
how  little  work  has  been  done  In  the  field. 

40,  Determining  the  Maximum  Value  of  a  Function 

Let  us  begin  with  a  simple  deterministic  problem.  Given  a 
continuous  function  f(x)  defined  over  the  Interval  [p,l2  , 
we  wish  to  determine  the  value  of  x  which  maximizes  f(x). 

For  a  variety  of  reasons,  some  of  which  we  have  discussed 
above,  we  do  not  wish  to  use  calculus,  but  wish  rather  to 
employ  a  search  method.  To  make  the  problem  of  determining  the 
value  of  a  maximizing  x  In  an  efficient  fashion  precise,  let 
US  pose  the  following  problem. 

"Given  the  continuous  function  f(x)  defined  over  J3,lj  , 
<leterBlne  the  quantity  N(d)  and  the  associated  search  policy 
ao  that  one  can  guarantee  that  a  maximum  value  can  be  located 
within  an  Interval  of  length  d  In  at  most  N(d)  steps." 
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If  t^ e  function  Is  taken  merely  to  te  continuous,  with  no 
additional  properties.  It  la  clear  that  N(d)  -  1/d.  If, 
however,  we  add  that  f(x)  Is  concave,  then  this  number  can  be 
considerably  reduced,  and  the  protlem  possesses  a  very  elegant 
solution . 

Ti  ls  solution  was  found  first  by  Kiefer,  [4^  ,  and  then. 
Independently,  and  In  a  simpler  fashion  by  Johnson,  [^  ,  using 
functional  equations. 

A  slrr.llar  problem  can  be  posed  with  reference  to  locating 
the  unique  zero  of  a  continuous,  monotone,  concave  function. 
Tbils  has  been  resolved,  using  functional  equations,  by  Gross 
and  Johnson  ,  [3^  . 

A  detailed  discussion  of  this  type  of  problem  will  be 
found  In  Kiefer,  . 

41.  Sequential  Testing 

The  problem  we  have  discussed  In  the  preceding  section  Is 
a  particular  case  of  the  general  problem  of  sequential  testing. 
Let  us  discuss  two  particular  problems  which  will  Illustrate 
the  difficulties  In  this  domain. 

Suppose  that  we  have  a  piece  of  equipment  which  has  N 
different  parts  to  be  exairdned  If  there  Is  loss  of  function. 
Given  a  priori  protablllty  distributions  associated  with  the 
Individual  parts,  and  a  set  of  testing  devices  which  furnish 
various  Indications,  how  should  we  proceed  so  as  to  locate  all 
sources  of  malfunction  In  a  minimum  time'?’ 

One  version  of  this  problem  arises  In  connection  with  de— 
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If  t^ e  function  Is  taken  merely  to  te  continuous,  with  no 
additional  properties.  It  la  clear  that  N(d)  -  1/d.  If, 
however,  we  add  that  f(x)  Is  concave,  then  this  number  can  be 
considerably  reduced,  and  the  protlem  possesses  a  very  elegant 
solution . 

Ti  ls  solution  was  found  first  by  Kiefer,  [4^  ,  and  then. 
Independently,  and  In  a  simpler  fashion  by  Johnson,  [^  ,  using 
functional  equations. 

A  slrr.llar  problem  can  be  posed  with  reference  to  locating 
the  unique  zero  of  a  continuous,  monotone,  concave  function. 
Tbils  has  been  resolved,  using  functional  equations,  by  Gross 
and  Johnson  ,  [3^  . 

A  detailed  discussion  of  this  type  of  problem  will  be 
found  In  Kiefer,  . 

41.  Sequential  Testing 

The  problem  we  have  discussed  In  the  preceding  section  Is 
a  particular  case  of  the  general  problem  of  sequential  testing. 
Let  us  discuss  two  particular  problems  which  will  Illustrate 
the  difficulties  In  this  domain. 

Suppose  that  we  have  a  piece  of  equipment  which  has  N 
different  parts  to  be  exairdned  If  there  Is  loss  of  function. 
Given  a  priori  protablllty  distributions  associated  with  the 
Individual  parts,  and  a  set  of  testing  devices  which  furnish 
various  Indications,  how  should  we  proceed  so  as  to  locate  all 
sources  of  malfunction  In  a  minimum  time'?’ 

One  version  of  this  problem  arises  In  connection  with  de— 
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these  problems,  we  encounter  very  much  the  same  difficulty  that 
we  met  In  the  traveling  salesman  proLlem.  As  we  proceed  In  our 
testing,  the  Infonr.atlon  pattern  tecomes  tremendously  complica¬ 
ted,  and  It  appears  to  be  Impossible  to  describe  the  state  of 
the  system  In  any  simple  way. 

43.  Design  of  Experiments 

The  problems  presented  In  the  foregoing  sections  are  In 
turn  special  cases  of  what  may  he  called  the  general  problem  of 
the  design  of  experiments. 

We  have  considered  Initially  the  relatively  simple  case  In 
which  the  structure  of  the  system  Is  assumed  known.  If  the 
structure  Is  taken  to  be  partially  unknown,  we  first  encounter 
situations  In  which  we  hypothesize  a  stochastic  structure,  and 
then  the  very  much  more  difficult  situations  In  which  we  have 
to  determine  the  structure  on  the  basis  of  observation  as  we 
proceed . 

The  Information  we  possess  determines  the  decisions  we 
make,  and  the  decisions  we  make  determine  the  new  Information 
pattern.  The  problem  of  detemilnlng  optimal  policies  In 
situations  of  this  type  Is  very  much  more  difficult  than  any  of 
the  problems  we  have  previously  discussed. 

Not  only  Is  the  analysis  much  more  Intricate  because  of 
the  stochastic  structure  of  the  process,  but  It  Is  no  longer 
easy  to  make  precise  what  we  mean  by  an  optimal  policy. 

For  a  detailed  discussion  of  matters  of  this  nature,  we 
refer  to  the  papers  by  Robbins,  [4?],  Bellman  and  Kalaba,  [22], 
Bellman,  [14]  ,  and  Karlin,  [39],  Karlin  and  Johnson,  [3S]  . 
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